FIG. 1 shows an optical communication system 100 that can use the invention. The system 100 includes laser and modulator 110 for converting a bit stream m(t) 101 at a predetermined bit rate to analog pulses of light 102. An optical fiber 120 carries the light pulses to a photo-diode 130 that converts the light pulses back to a signal 103. An amplifier/receiver 140 amplifies and filters the signal 103 to remove wideband noise and to produce a received signal x(t) 104. The bit stream is sampled at the bit rate. The samples are compared with a sampling threshold. If a sample of x(t) is greater than the threshold, then it is a one, otherwise it is a zero. This recovers the input bitstream 101.
In practical systems, the received signal 104 is distorted in time and shape leading to a degradation known as inter-symbol interference (ISI) when pulses overlap or otherwise interfere with each other. ISI increases the bit error rate (BER) in the system 100. If the ISI is severe, an equalizer can compensate for the distortion in the received signal x(t) 104. Ideally, it is desired to remove dispersive effects completely, and to recover the original bit stream 101 with a minimal number of errors.
In one possible solution, the receiver 140 reduces the ISI using a compensator, commonly called an equalizer. There are many types of equalizers used in practical digital communication systems, such as maximum-likelihood (ML) estimation based equalizers, linear filtering with adjustable coefficients, decision feedback equalizers (DFE), etc., see Proakis, Digital Communications, Fourth Edition, McGraw-Hill, New York, 2001. In order to be used for unknown channels, the equalizers are automatically adjusted to the channel impulse response and time variations in the channel impulse response. This technique is called adaptive equalization.
Adaptive equalizers can be implemented as digital filters that operate on quantized samples of the received electrical signal 104. The difficulty with digital equalization is that the received signal needs to be converted to digital form before it can be filtered, and then the digital signal needs to be sampled and quantized at least once per symbol period.
Optical communication systems can operate at bit rates greater than 10 Giga bits per second. This requires high-speed analog-to-digital converters and very fast digital equalizers, which is problematic.
FIG. 2 shows a typical prior adaptive equalizer 200 that includes a linear finite impulse response (FIR) filter 210 with adjustable weight coefficients, and a decision device 220. An input signal 201 to the equalizer 200 is an ISI distorted version of transmitted signal with noise. The weights are adjusted according to an error signal e(n) 204 produced by an adder 230, which takes as input the signals the output y(n) of the FIR 210 and the input d(n) of the decision device 220.
A variety of techniques have been developing to adaptively adjust the weights of the filter 210 in a manner that minimizes the distortion. One commonly used error metric is a peak distortion error. This is the well-known zero-forcing equalizer described by Proakis. Although this error metric reduces ISI, it significantly increases noise.
To alleviate the increase in noise, a metric based on the mean-square error (MSE) can be used. In this case, the error metric is defined as E[d(t)−y(t)2] The operator E[ ] is the expectation operator, d(t) is the desired response of the equalizer 210, and y(t) is the actual response seen at the output of the FIR filter 210. A least-mean squares error (LMS) can be used to determine the filter parameters that minimize the MSE. For an LMS-type equalizer, the tap weight coefficients of the equalizer are adjusted recursively by the following:w1(n+1)=w1(n)+μe(n)·x1(n) i=1, . . . , N,  (1)
where N is the equalizer length, w=[w0, w1, . . . wN], is a tap weight vector of length N, n is a time index, μ is a step size parameters that controls the rate of convergence, e(n) is the error signal 204, and xi is a delayed version of the input signal 201. The error signal 204 is defined bye(n)=d(n)−y(n),  (2)
where y(n) is the output signal 202 of the adaptive equalizer and d(n) is the output of the decision device.
In general, the output of the decision device 220 can be one of many complex levels. Most optical communications systems use a simple two level amplitude shift keying technique, known as on-off keying. In that case, the output of the decision device is one of two levels, a high level representing a one bit or a low level representing a zero bit. However, because optical communication systems can operate at bit rates greater than 10 Giga bits per second high-speed analog-to-digital converters and very fast digital equalizers are problematic.